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FLYING  TRAINING  R&D  AT  THE  AIR  FORCE  HUMAN  RESOURCES  LABORATORY 


INTRODUCTION 

Flying  Training  research  and  development  (R&D)  In  the  United  States  Air 
Force  Is  carried  out  principally  by  the  Air  Force  Human  Resources  Laboratory 
(AFHRL)  which  has  four  major  locations  in  the  United  States.  The  Laboratory 
Headquarters,  the  Manpower  and  Personnel  Division,  and  the  Technical  Services 
Division  are  located  at  Brooks  AFB,  San  Antonio,  Texas.  The  Operations 
Training  Division  and  the  Logistics  and  Human  Factors  Division  are  located  at 
Williams  AFB,  Phoenix,  Arizona,  and  Wrlght-Patterson  AFB,  Dayton,  Ohio, 
respectively.  The  Training  Systems  Division  is  located  at  Lowry  AFB,  Denver, 
Colorado.  Each  of  these  Divisions  is  colocated  with,  or  located  very  near, 
the  users  of  its  R&D  products. 

The  mission  of  AFHRL  Is  to  conduct  R&D  in  the  areas  of  Selection  and 
Classification,  Logistics  and  Technical  Training,  and  Flying  Training. 
Selection  and  Classification  R&D  addresses  recruitment,  selection  and  assign¬ 
ment,  productivity,  and  retention.  Logistics  R&D  focuses  on  the  productivity 
of  maintenance  personnel  and  on  the  specification  of  logistics  requirements 
during  the  weapons  systems  acquisition  process.  Technical  Training  R&D 
concentrates  on  the  design  of  maintenance  simulators  and  on  the  development  of 
computer  and  Instructional  technologies  to  Improve  Air  Force  resident  school 
and  on-the-job  training.  Flying  Training  R&D  centers  on  developing  new 
simulation  and  training  device  technologies  for  application  to  aircrew 
training  programs. 


FLYING  TRAINING  PROGRAM 


Simulator  Effectiveness 


Much  of  the  Laboratory  R&D  In  Flying  Training  Is  performed  at  Williams 
AFB  and  uses  the  AFHRL  Advanced  Simulator  for  Pilot  Training  (ASPT).  This 
simulator,  which  became  operational  In  1975,  consists  of  two  cockpits  mounted 
on  slx-degrees-of -freedom  motion  platforms  with  wide-angle  Infinity  optics 
visual  displays  and  a  black-and-white  computer-generated  Image  of  the  flight 
environment.  The  cockpits  are  currently  configured  as  A-10  and  F-16  aircraft 
and  can  simulate  all  aspects  of  flight  training.  Including  a  limited  simula¬ 
tion  of  air-to-air  and  alr-to-surface  combat  scenarios.  An  Instructor 
operator  station  permits  problem  Insertion,  subject  monitoring,  and  perform¬ 
ance  measurement. 

Some  of  the  Flying  Training  R&D  performed  by  AFHRL  has  advanced  the 
state-of-the-art  In  flight  simulator  hardware.  Projection  devices  and 
computer  Image  generation  (CIG)  techniques  have  been  developed  to  Improve 
simulator  scene  content  for  training  pilots  In  realistic  air-to-air  and 
alr-to-surface  environments.  Ongoing  R&D  will  add  color,  more  scene  detail, 
better  resolution,  and  additional  offensive  and  defensive  air-to-air  and 
a Ir-to-surf ace  threats  to  the  ASPT  visual  scene.  Other  enhancements  will 
Include  simulated  air-to-air  combat  options  of  1  versus  1,  2  versus  1,  and  2 
versus  2.  Each  of  these  enhanced  CIG  features  Is  being  integrated  Into  the 
ASPT  F-T6  configuration. 
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Initially,  the  behavioral  R&D  conducted  on  the  ASPT  supported  the  Air 
Force  Undergraduate  Pilot  Training  ( UPT)  program.  Contributions  were  made  to 
the  development  of  the  UPT  flying  training  syllabus  and  the  Instrument  Flight 
Simulator  training  syllabus.  Several  studies  have  also  been  conducted  on  the 
transfer  of  training  effectiveness  of  platform  motion.  Based  on  a  review  of 
six  such  studies,  Martin  (1981)  concluded  that  platform  motion  cueing  results 
in  negligible  transfer  of  training  for  Initial  jet  piloting  skills.  She  also 
concluded  that  existing  (1980)  platform  systems  would  not  significantly 
increase  the  transfer  of  training  of  tasks  that  call  for  more  advanced  pilot¬ 
ing  skills,  such  as  air-to-air  combat  and  nap-of-the-earth  flight.  All 
studies  reported  by  Martin  employed  a  wide  f ield-of-view,  black-and-white 
visual  display  with  a  six-degrees-of-freedom  motion  platform. 

More  recent  behavioral  R&D  has  focused  on  advanced  air  combat  tactics 
training,  rather  than  on  UPT.  For  example,  ASPT  A-10  and  F-16  simulations 
have  demonstrated  positive  transfer  of  training  from  the  A-10  ASPT  to  the 
gunnery  range  (Gray,  Chun,  Warner,  &  Eubanks,  1981).  Seventeen  pilots  were 
trained  In  air-to-ground  weapons  delivery  In  the  simulator  and  then  were 
tested  on  the  actual  gunnery  range.  In  conventional  dive  bombing  tasks, 
simulator-trained  pilots  had  a  circular  error  of  75  feet,  whereas  pilots  with 
no  pretraining  (N«7)  had  an  average  error  of  94  feet.  Instructor  pilots  had 
an  average  error  of  only  56  feet,  but  In  some  cases  were  outscored  by  their 
simulator- trained  students.  In  the  strafing  task,  61%  of  the  rounds  fired  by 
the  simulator-trained  group  hit  the  target,  as  compared  to  only  41X  of  the 
rounds  fired  by  the  non-simulator- trained  groip. 

The  results  on  the  gunnery  range  suggested  the  value  of  extending 
simulation  training  to  full  combat  training.  A  first  effort  which  demon¬ 
strated  that  simulators  could  be  used  for  this  extended  role  was  reported  by 
Kellogg,  Prather,  and  Castore,  (1980).  The  ASPT  visual  scene  was  modified  to 
depict  a  hostile  environment  complete  with  mountains,  hills,  flatlands,  and 
enemy  weaponry,  including  anti-aircraft  artillery  and  surface-to-air  missiles 
(SAMs)  at  strategic  locations.  The  pilot  could  see  muzzle  flashes  from  the 
anti-aircraft  batteries,  which  had  a  kill  probability  of  100%  if  the  pilot 
allowed  one  to  achieve  a  tracking  solution  for  6  seconds.  A  pilot  who  flew 
within  the  firing  envelope  of  a  SAM  would  hear  a  warning  tone  from  the 
electronic  warfare  equipment  and  see  the  missile  in  flight.  The  SAM  could  be 
evaded  with  proper  maneuvering.  The  pilot's  task  was  to  fly  into  the  hostile 
area,  locate  and  destroy  a  tank,  and  fly  out  safely.  Seven  combat  qualified 
pilots  served  as  subjects,  and  their  performance  was  evaluated  in  terms  of 
whether  they  found  and  destroyed  the  tank  and  flew  out  of  the  hostile  area 
safely.  By  the  conclusion  of  the  first  10  trials,  the  target  was  being 
destoyed  more  than  80%  of  the  time,  but  only  60%  of  the  pilots  survived. 

After  20  trials,  the  number  of  targets  destroyed  remained  the  same,  but  the 
survival  rate  increased  from  60%  to  80%.  With  experience,  pilots  were  able  to 
attend  more  to  the  defensive  task.  The  study  clearly  indicated  that  combat 
tasks  can  be  trained  in  a  simulator. 

Another  study  (Hughes,  Brooks,  Graham,  Sheen,  &  Dickens,  1982)  took  the 
next  logical  step  of  assessing  whether  combat-like  simulator  training 
resulted  in  inproved  performance  in  the  aircraft.  Eleven  A-10  pilots 
participating  In  an  Air  Force  RED  FLAG  war  game  were  given  training  in  the 
ASPT  prior  to  flying  the  RED  FLAG  exercise.  Training  consisted  of  both 
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interdiction  and  close-air-support  Missions  in  a  simulated  electronics  warfare 
environment  where  the  threat  approximated  that  of  a  typical  enemy  air  defense 
system.  The  11  pilots  with  simulator  training  survived  89%  of  the  total 
sorties  flown,  whereas  those  without  pretraining  survived  only  75%  of  the 
sorties.  These  results  provided  the  first  evidence  of  transfer  of  training 
from  the  simulator  to  the  aircraft  under  combat  conditions. 

Part-Task  Trainers 


In  addition  to  conducting  transfer-of-training  studies  on  full  mission 
simulators,  AFHRL  scientists  have  also  been  investigating  the  effectiveness  of 
part-task  trainers.  In  one  transfer  of  training  study,  Nullmeyer  and  Laughery 
(1980)  evaluated  a  B-52  air  refueling  part-task  trainer  (ARPTT)  that  consisted 
of  a  cockpit,  visual  system,  platform  motion  system,  and  instructor  operator 
station.  The  subjects  were  98  Air  Force  pilots  undergoing  training  for 
certification  or  recertification  as  B-52  aircraft  commanders.  All  were  either 
current  B-52  co-pilots,  former  B-52  aircraft  commanders,  or  pilots  cross¬ 
training  from  a  different  aircraft.  Without  training  on  the  ARPTT,  the 
co-pilots  required  an  average  of  9.9  in-flight  sorties  to  reach  proficiency. 
When  trained  to  proficiency  on  the  ARPTT,  however,  they  required  only  5.8 
aircraft  sorties  to  reach  in-flight  proficiency.  Pilots  transitioning  from 
other  aircraft  showed  a  reduction  from  10.6  sorties  to  9.2  sorties,  and  former 
B-52  commanders  had  a  reduction  from  4.6  sorties  to  3.4  sorties.  In  all 
cases,  proficiency  was  assessed  by  Instructor  pilots  using  standardized  rating 
forms. 

In  another  study,  Pohlman  and  Edwards  (1983)  investigated  the  effec¬ 
tiveness  of  a  microcomputer-based  desk-top  trainer  used  to  teach  F-16  cockpit 
stores  management.  Cockpit  stores  management  involves  selecting,  arming,  and 
firing  a  broad  spectrum  of  aircraft  weaponry  by  pressing  buttons  and  Inter¬ 
preting  displays.  An  experimental  group  learned  to  perform  alr-to-surface 
weapons  delivery  tasks  using  a  part-task  trainer  that  consisted  of  a  computer- 
assisted  Interactive  graphics  display,  while  a  control  group  received  the  same 
training  using  an  Illustrated  programmed  text.  After  training,  both  groups 
were  tested  on  their  ability  to  perform  similar  tasks  on  the  actual  F-16  stores 
control  panel.  The  experimental  group  was  able  to  complete  the  task  In  signi¬ 
ficantly  less  time  and  with  fewer  errors  than  the  control  group,  thus 
supporting  the  conclusion  that  Inexpensive  microcomputers  coupled  with  computer 
graphics  systems  can  effectively  provide  self-instructional.  Interactive 
training  to  aircrews  for  selected  procedural  tasks. 

By  combining  the  best  features  of  the  large,  full  mission  simulators  and 
the  smaller  portable  part-task  trainers,  AFHRL  Is  now  developing  a  helmet- 
mounted  display  that  can  be  used  to  simulate  complex  wartime  missions.  Called 
the  Combat  Mission  Trainer,  It  Is  intended  to  augment  or  replace  the  large, 
expensive  wraparound  mosaic  or  dome  type  displays  now  in  use.  The  afford¬ 
ability  and  transportability  of  a  Combat  Mission  Trainer  will  make  ft  possible 
to  train  combat  missions  at  the  squadron  level  and  to  deploy  the  simulator 
with  the  squadron.  If  necessary. 


Performance  Measurement 

The  development  of  automated  aircrew  performance  measures  has  been  a 
continuing  effort  at  AFHRL  since  the  1960's.  Early  work  centered  on  the 
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measurement  of  performance  In  the  T-37  aircraft.  Measures  were  developed  for 
takeoff,  landing,  formation  flight,  instrument  flight,  air  refueling,  and 
basic  aerobatics.  The  automated  system  for  measuring  pilot  performance  in  the 
ASPT  is  described  In  Fuller,  Waag,  and  Martin  (1980). 

Two  programs  now  under  development  are  the  C-5A  aircrew  performance 
measurement  system  and  the  air-to-air  combat  measurement  system.  The  C-5A 
system  will  objectively  measure  and  score  all  aspects  of  individual  and  flight 
crew  performance  in  the  areas  of  checklists  and  procedures,  aircraft  state 
parameters,  and  navigational  profiles.  Approximately  3,000  separate  measures 
can  be  taken,  and  weights  assigned  to  these  measures  based  on  criticality  of 
the  performance  for  the  particular  maneuver  being  flown.  Algorithms  now  being 
developed  will  combine  tne  Individual  measures  and  derive  overall  mission 
assessments.  When  completed,  the  measurement  system  will  be  installed  in  the 
C-5A  simulator  and  also  in  the  C-5A  aircraft,  thereby  allowing  comparisons  to 
be  made  of  aircrew  performance  in  both  the  simulator  and  aircraft. 

In  measuring  air-to-air  combat  performance,  the  approach  has  been  to 
relate  aircraft  handling  variables  to  number  of  engagement  kills.  In  a  study 
by  Kelly,  Wooldridge,  Hennessy,  Vreuls,  Barnebey,  Cotton,  and  Reed  (1979),  28 
such  measures  were  related  to  air  combat  maneuvering  experience.  Thirty 
subjects  were  divided  Into  three  groups  based  on  experience  level:  students 
with  only  basic  fighter  maneuver  training,  students  in  advanced  training,  and 
air  combat  Instructor  pilots.  During  405  air  combat  engagements  in  the 
simulator,  the  basic  and  advanced  students  had  about  20ft  kills,  whereas  the 
Instructor  pilots  had  over  40*  kills.  Each  subject  was  measured  on  28 
variables  to  determine  how  the  performance  of  the  high  and  low  skill  groups 
differed.  Multivariate  discriminant  analyses  were  performed,  and  results  were 
incorporated  into  an  algorithm  containing  13  measures  including  altitude 
change  rate,  speed  brake  and  throttle  usage,  offensive  time,  and  energy 
management.  These  measures  accounted  for  51ft  of  the  variance  in  the 
performance  data  and  discriminated  between  members  of  the  high  and  low  skill 
level  groups  with  92ft  accuracy.  While  this  and  other  approaches  examined  have 
been  useful  for  assessing  overall  maneuvering  performance,  they  have  not 
provided  good  diagnostic  Information.  Measures  are  still  needed  to  define 
what  went  wrong,  where  It  went  wrong,  and  why.  This  information  can  then  be 
Incorporated  Into  flight  screening  and  training  programs. 

Selection  Classification 

Current  R&D  on  aircrew  selection  and  classification  focuses  on  developing 
new  UPT  selection  devices.  Presently,  the  UPT  screening  decision  is  based  on 
Air  Force  Officer  Qualifying  Test  scores,  medical  examinations,  and  light 
aircraft  (T-41)  flying  evaluations.  Additional  measures  now  being  developed 
at  AFHRL  for  better  prediction  of  success  In  training  and  in  the  operational 
flying  environment  Include  psychomotor  tests  and  measures  of  a  candidate's 
ability  to  process  Information  and  make  decisions  while  performing  complex 
flying  tasks.  These  new  measures  and  others,  based  on  training  grades  and 
class  rank,  will  be  evaluated  against  flying  performance  In  both  the  training 
and  operational  environments.  Initial  analyses  of  data  are  encouraging  and 
suggest  that  the  new  measures  can  be  used  to  predict  better  whether  a  student 
will  pass  or  fall  UPT.  The  measures  will  also  be  assessed  for  their  effec¬ 
tiveness  in  screening  pilots  for  assignment  to  a  fighter-attack-reconnaissance 


pilot  training  track  or  a  tanker-tran sport-bomber  track,  following  a  common 
primary  phase  of  training.  This  R&D  is  in  response  to  the  planned  inaugu¬ 
ration  of  a  Specialized  Undergraduate  Pilot  Training  program,  comnonly 
referred  to  as  a  "dual-track*  flying  training  program,  that  will  begin  in  1986. 

CONTINUING  R&D  ISSUES 

The  foregoing  is  by  no  means  a  complete  summary  of  all  the  R&D  conducted 
at  AFHRL  in  the  Flying  Training  area  over  the  past  several  years.  It  is  only 
a  sample  of  studies  recently  completed  or  now  underway.  Although  Inclusion  of 
additional  studies  would  add  to  the  overall  R&D  picture,  the  conclusion  would 
remain  the  same:  more  data  are  required  to  adequately  answer  the  numerous 
selection,  classification,  and  training  questions  being  asked  daily  by  Air 
Force  managers,  engineers,  and  training  specialists. 

Despite  years  of  R&D  by  behavioral  scientists  in  government,  industry, 
and  universities,  perennial  questions  remain.  What  type  person  makes  the  best 
fighter  pilot?  Transport  pilot?  Bomber  pilot?  How  quickly  do  flying  skills 
decay?  Can  these  skills  be  retrained  best  in  a  simulator  or  In  an  aircraft? 

At  what  cost  differential?  What  degree  of  simulator  fidelity  is  required? 

The  answers  to  these  questions  are  not  simple.  Yet,  behavioral  scientists 
must  answer  them  or  training  courses  and  training  devices  will  continue  to  be 
developed  only  In  terms  of  engineering  design  criteria  and  expert  opinion 
rather  than  In  terms  of  training  effectiveness  criteria  based  on  experimental 
data.  Moreover,  program  managers  will  continue  to  assign  low  priorities  to 
behavioral  science  R&D  programs. 

What  may  be  required  to  provide  more  comprehensive  answers  to  these 
questions  is  a  concerted  effort  by  multidisciplinary  teams  working  together  on 
well  defined  practical  problems.  These  team  efforts  should  have  a  higher  pay¬ 
off  than  dispersed  Individual  efforts  that  often  address  only  very  small 
percentages  of  the  variance  associated  with  the  total  behavior  in  question. 

The  knowledge  of  multidisciplinary  teams  often  permits  a  broader  perspective 
of  the  problem.  A  practical  orientation  Is  also  required.  For  Air  Force 
exploratory  and  advanced  development  projects,  the  potential  for  a  practical 
payoff  must  be  demonstrated  before  the  project  is  approved  and  funded.  The 
problem  must  also  be  well  defined  and  thoroughly  related  to  already  published 
data  so  that  the  potential  for  generalization  is  Increased.  General izability 
is  the  goal;  specificity  is  the  rule.  Too  many  behavioral  studies  have  not 
only  had  little  value  for  practical  application,  but  have  also  had  little 
relationship  to  any  theoretical  knowledge  base.  The  results  stand  alone. 

As  the  Air  Force  lead  organization  for  conducting  Flying  Training  R&D, 
AFHRL  frequently  calls  upon  the  talents  of  universities  and  industrial  organ¬ 
izations  for  assistance.  The  usual  method  is  through  the  solicited  proposal 
process,  although  unsolicited  proposals  are  also  welcome.  Two  other  means  of 
participating  In  Air  Force  R&D  programs  are  through  a  10-week  Summer  Faculty 
Research  Program  (SFRP)  for  faculty  and  graduate  students  and  through  the 
Intergovernmental  Personnel  Act  (IPA)  of  1970  which  permits  1-year  appoint¬ 
ments  of  scientists  to  Air  Force  laboratories.  The  SFRP  is  sponsored  by  the 
Air  Force  Office  of  Scientific  Research  (AFOSR)  and  conducted  by  the 
Southeastern  Center  for  Electrical  Engineering  Education  (SCEE).  Detailed 
Information  on  application  procedures  can  be  obtained  from  SFRP  Program 
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Director,  SCEE  Management  Office,  1101  Massachusetts  Avenue,  St  Cloud,  Florida 
32769.  The  1984  program  runs  between  IS  May  and  30  September  1984  with  a 
flexible  start  date  for  the  10-week  session.  The  graduate  student  research 
period  must  coincide  with  the  appointment  period  of  the  supervising  professor 
with  whom  the  student  is  working.  The  IPA  appointments  are  sponsored  by  AFOSR 
at  Bolling  AFB,  Washington  DC  20332,  and  by  individual  laboratories.  Persons 
seeking  IPA  appointments  with  AFHRL  should  submit  inquiries  to  AFHRL/XR, 

Brooks  AFB,  Texas  78235.  Applications  for  both  the  IPA  program  and  the  SFRP 
are  welcome. 
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